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ABSTRACT 



This study extends earlier research on headings use in 
undergraduate testing, and examined whether headings within multiple-choice 
tests positively impact test performance. Participants were employees (n=143) 
of a large manufacturing plant in the western United States. The experimental 
group received a multiple-choice recertification test with headings included; 
the control group received the same test without headings. Findings indicated 
that headings within multiple-choice tests did positively affect test 
performance and suggest that headings may differentially impact test items of 
differing difficulty. Future research should focus on increasing heading 
specificity. (Contains 27 references.) (AEF) 
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Background 

There is a substantial research base considering the effects of headings within text, however there are almost 
no studies involving the use of headings during testing. Headings in text research results have been mixed. 
Headings in text appear to affect the encoding of certain kinds of information including declarative information and 
recall of concepts, but has been shown to be nominally effective for recall of logical relations. Also, lessons 
including headings seem to benefit less able readers, older readers, and field dependent readers. Learner-generated 
headings positively impact delayed recall of inference especially for higher-ability learners. 

The effects of headings during testing were investigated by Townsend, Moore, Tuck, and Wilton (1990). A 
sample of undergraduate students (n = 287) enrolled in an introductory educational psychology course received a 41 
item multiple-choice examination during the first quarter of the course that was 20% of their final course grade. The 
students were randomly assigned to receive either the version with headings or the version without headings. The 
headings were based on the title of the chapters covered during each part of the course. Field dependence-independence 
and student attitude towards the use of headings during testing were also considered. Headings did not facilitate (or 
hinder) test performance and no association was observed between field dependence-independence and headings. 
Interestingly, the students overwhelmingly reported that they felt that headings during testing would positively 
influence their performance. 



Aims 

Do headings within multiple-choice tests impact test performance? It has been suggested that headings 
within multiple-choice tests positively affect test performance, however, there is little research to date to either 

support or reject this hypothesis. This study ("^'Marcinkiewicz & Clariana, 1997) replicates and extends Townsend et 
al. (1990) by utilizing a different population and by including a covariate to increase statistical power. 

Sample 

The available sample for this study included employees of a large manufacturing plant in the western part of 
the United States (n=143). Participants ranged in age from 26 to 64 years old, the median age was 44 years old. 

Method 

Participants were randomly assigned to the experimental or control treatments. The experimental group 
received a multiplerchoice recertification test with headings included; the control group received the same test without 
headings. The test consisted only of questions; no text passages were included. The multiple-choice test items were 
developed by a panel of subject-matter experts (SMEs) and test specialists. The items were pilot-tested and revised 
by a second group of SMEs. The final test version was approved by SMEs and plant management. The test 
included five content areas: Mathematics, Physics and Nuclear Physics, Radiation Biology and Exposure Control, 
Dosimetry, and Radiation Instrumentation. The KR-20 reliability was 0.79. 

The test items were grouped by content area. The words used as headings for each item grouping were the 
same as the content areas listed above. This type of heading is approximately equivalent to the use of chapter titles 
as headings in the Townsend etal. (1990) study. 



Results 

The data were analyzed by analysis of covariance (ANCOVA), a second similar test served as the covariate. 
A significant difference was obtained for the treatment main effect, F(l, 140) = 4.33 (p < .04), with a calculated 
effect size for headings (x = 76.4) over no headings (x = 74.7) of es = 0.33 (adjusted). A post hoc analysis of the 
influence of headings on test items of differing difficulty was conducted. This analysis indicates that headings in 
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tests have little effect on difficult test items, but a powerful effect on easier items (see Table 1). Table 1. Impact of 
headings by test item difficulty. 



item difficulty: 


difficult 


moderate 


easy 


No-headings eroup 


X = 


51.8 


73.5 


89.8 




s.d. 


(10.5) 


(8.7) 


(3.8) 


Headings group 


X = 


51.6 


11. Z * 


93.3 ♦ 




s.d. 


(6.6) 


(7.4) 


(2.8) 


effect size 




0.02 


0.44 


0.92 



Note: * significant at p<0.05 level; No-headings group, n of subjects = 60, 
Headings group, n of subjects = 83 



Conclusions 

Contrary to a similar earlier study by Townsend etal. (1990), the findings of the present study indicate that 
headings within multiple-choice tests positively affect test performance. As in the Townsend study, the headings 
used in the present study apply at the "macro" level of the tested content. In both studies the tests were relatively 
important to the participants. Though the participants in both studies were adults, the populations were very 
different. Also, the use of ANCOVA in the present study allowed for increased power of discrimination compared to 
the earlier study. 

Perhaps most important, the findings of this study suggest that headings may differentially impact test 
items of differing difficulty. Thus, uncontrolled item difficulty effects may account for the lack of significance in the 
Townsend study. Further, previous mixed results for headings in text may be due to uncontrolled item difficulty 
effects. A meta- analysis of headings in text studies should consider item difficulty. Future headings within test (and 
text) research should control item difficulty. 

It is advised that selecting a ppropriate headings to use during testing is important. Headings that are highly 
specific to each item could reasonably be expected to obtain a greater effect than the generalized or "meta" level of 
headings used in this study. Future research should focus on increasing heading specificity. Possibly learner 
generated headings may provide the highest level of specificity with the greatest effect. 
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